Journal of Vision

Blinks are a ubiquitous yet largely unnoticed aspect of human vision, despite causing frequent interruptions of visual input that can amount to up to 10% of waking time. By leveraging a large dataset of unconstrained gaze behavior during two natural tasks, we found a novel behavioral strategy to limit the impact of blinks: blinks were strategically coupled with head movements, which minimizes information loss due to unreliable visual input during head movements. Specifically, blink probability increased with higher head velocities and showed strong temporal modulation relative to head movement onset. Blink probability was reduced before head movement initiation and then peaked during the head movement. The strength of this coupling was tailored to the individual needs of participants, with participants with higher baseline blink showing a stronger synchronization. This indicates that blinks are a part of an individually coordinated strategy when orchestrating eye and head movements during unconstrained natural behavior.

The ability to mentally rotate objects is a fundamental feature of human cognition, and humans can use this ability to make choices about objects based on their geometry. However, remarkably little is known about how such choices are reached, and what sort of visual information might facilitate them. We devised an experiment where participants had to mentally simulate an object's rotation to choose which of two objects was better for a subsequent task based on its shape alone. We also tracked their gaze while they made their choice, to see which visual information they were using to facilitate this mental simulation. We found that participants were consistently able to choose the most suitable object for the task, and, remarkably, the visual information they sampled was directly linked to their choices. Put simply, participants made better choices when they looked at more informative regions of the objects, and participants who sampled regions that were better for facilitating mental simulation made better choices overall. These findings reveal a direct link between fixations, simulation, and decision-making, suggesting that to perform any fine-grained mental simulation people need to direct their gaze at specific, informative points of an object to simulate its two-dimensional proximal image displacement.

Representations of the same physical stimulus vary between individuals. Characterizing individual differences has practical implications, but is challenging because these representations are not directly observable. Given a model of how representations vary within a population, we propose a Bayesian adaptive procedure for estimating an individual observer's representation from a series of targeted perceptual discrimination judgments. A key component of our approach is using Fisher information to identify stimulus distortions that efficiently differentiate observers in the population. As a proof of concept, we focus on individual differences in color perception and simulate observers with cone fundamentals drawn from an individual colorimetric observer model. We demonstrate that our approach can recover key aspects of a sampled observer's cone fundamentals using simulated three-alternative forced-choice oddity judgments with approximately 500 trials, corresponding to an experimental duration of approximately one hour. Our Bayesian adaptive framework provides a promising and generalizable approach to efficiently link behavioral measurements to individual differences in sensory representations.

During sideways movement of an observer, optic flow parsing - in which an objects speed in the world is extracted from all the other visual movement present in the scene, self-generated and otherwise - has been shown to be incomplete, leading to biases in speed perception, particularly when object and observer are moving in opposite directions. Here, we assess how judgements about the speed of objects moving in depth (judged relative to the world) towards or away from an observer (6 m/s) are affected by simultaneous movement of the observer either in the same or opposite direction as the object. In a virtual reality display, participants (n = 25) viewed a sphere simulated as moving in a corridor either while they were stationary or during visually simulated self-motion in the same or opposite direction as the object. They judged the spheres movement relative to the world by comparing its motion to a probe sphere that travelled laterally across the corridor in front of them. In a second experiment (n = 28) participants performed the same task but during faster self-motion (10 m/s). The second cohort also judged the direction in which the object was perceived to move during the same combinations of self and object speeds. Object speed was overestimated when the object travelled in the direction opposite to the observer compared to how objects motion was judged when the observer was stationary. However, object speed was also overestimated during self-motion in the same direction as the object where participants were also much more likely to misjudge the direction of motion of the object. Precision of judgements was lower when self-motion was simulated than it was for stationary observers. A simple arithmetic model of flow parsing fails to capture these results satisfactorily, suggesting that different mechanisms may be at play when the observer travels in the same direction as a moving object and is vulnerable to misperceiving its direction of travel.

Current debates regarding the relative contribution of saliency versus semantics to gaze control often rely on comparing the predictive power of saliency and meaning maps. We argue that such indirect, global approaches are fundamentally limited because fixations arise from heterogeneous, local causes that are conflated in whole-scene comparisons. To substantiate this claim, we used a direct method where participants explicitly identified the reasons for fixation at specific clusters of high fixation density, distinguishing between low-level saliency and various semantic categories, as well as the most important one. The obtained judgments revealed that multiple factors contribute simultaneously to gaze control. Although their influence varied across fixation clusters, semantics generally dominated saliency. Notably, abstract semantic categories, particularly "unknown/unusual," proved important, highlighting the role of prior knowledge and novelty besides personal relevance in guiding attention. To interpret these findings in the context of existing models, we propose a framework distinguishing between processes highlighting interesting locations in the image from a sampling strategy translating this information into scanpaths. Within this framework, classic saliency and meaning maps are viewed as restricted inputs to the strategy, whereas deep learning-based models (e.g., DeepGaze IIE) are more general and may also implicitly encode aspects of the strategy itself. Consistent with this, we found that the predictive performance of DeepGaze IIE varied less significantly with the specific reasons for fixation than that of classic saliency and meaning map approaches.

During active vision, the brain must coordinate where to move the eyes with predictions about upcoming sensory input. Before each saccade, perception is enhanced at the upcoming fixation location, but whether this enhancement depends on expectations about target features remains unknown. Here, participants prepared a saccade to a cued location while reporting the presence and orientation of a brief visual target that appeared either at the saccade goal or at the opposite location. Feature expectation was manipulated across blocks by varying the probability of the two target orientations. Perceptual sensitivity (d') increased when targets were presented at the saccade goal, consistent with presaccadic enhancement, and was also higher for less expected features. However, these effects were independent: feature probability did not alter the magnitude of presaccadic enhancement. Moreover, presaccadic enhancement increased near saccade onset, whereas the advantage for less expected features weakened as movement onset approached. Saccade latency revealed a contrasting pattern. Visual targets presented at the saccade goal delayed movement initiation. This delay depended on feature probability, with longer latencies for unexpected than for expected features only when saccades were directed towards the target. This location-specific effect persisted after accounting for perceptual report, and the latency cost for unexpected features was reproduced in a follow-up experiment. Together, these findings show that feature probability enhanced sensitivity to unexpected information independently of presaccadic enhancement, while selectively delaying saccade initiation towards targets with unexpected features. This dissociation suggests that feature expectation modulates perception and action through functionally distinct forms of visual processing.

Fixational eye movements (FEMs) comprise the involuntary small scale eye motion conducted during fixation on a stationary stimulus. As a consequence, the visual information can be spread across multiple photoreceptors reducing the local signal-to-noise ratio. Yet, the signals transmitted by individual photoreceptors adapt to constant stimulation so that an entirely still scene would eventually fade from view. Because FEMs convert a stationary stimulus in the world to a temporally varying one on the retina, they can act to prevent this stimulus fading. Thus, FEMs can be understood as a sampling protocol than needs to be adjusted to the underlying processing circuitry. We analyse the impact of FEMs on the rate of information acquisition at the level of the retina for two common tasks of the human eye that typically go hand in hand: detection and localization. Here, we build a simple analytical model of visual perception, i.e. we subject a continuous receptor array to a stimulus moving across the retina as a consequence of FEMs with receptor excitations depending on past stimulation through a linear response function. Using Bayesian inference we quantify both the probability of detection and the accuracy of localization as a function of parameters controlling eye movements and stimulus. We find that localization of a stimulus is equivalent to the detection of the stimulus gradient. This allows us to discern optimal properties of eye movements for the respective tasks and provides a link between two typical psychophysical observables: detection thresholds and Vernier acuity. Our analysis suggests that typical human FEMs tend to facilitate localization at the expense of detection. Simply put, if you can see a stimulus you also know where it is. Finally, we propose a variety of experimental protocols to investigate the interplay between FEMs, detection, and localization with the potential of inferring intrinsic properties of an individuals visual system.

Visual signals from the two eyes merge gradually as they pass through the primary visual cortex (V1). Here we use a computational model of Macaque V1 to study the first stage of this integration along the magnocellular pathway, in layer 4C, aiming to infer neuroanatomical origins of binocular response. It is known that neurons in layer 4C are predominantly monocular, though some do exhibit varying degrees of binocularity. We find (1) the emergence of narrow binocular strips along borders of ocular dominance columns (ODC), a finding that aligns with experiments; (2) most consistent with data is when 10 - 30% of interactions near ODC boundaries are cross-columnar; and (3) feedback from layer 6 is largely monocular. These results were obtained through systematic hypothesis testing using a multiscale model that is orders of magnitude faster than its biologically-detailed predecessors. We propose that multiscale modeling can be an effective tool for bridging anatomy and function.

Recent work has brought to light a number of stimulus families whose perception is shaped by strong idiosyncratic biases. These biases differ significantly from one observer to the next, yet remain quite stable within observers when measured over multiple points in time, sometimes over months or even years. Nevertheless, we have previously shown that at least some of these biases undergo small but systematic changes over time. Although these temporal changes are also idiosyncratic, they generally act as a kind of memory that accumulates small random steps. Other research has shown that when stimuli are shown at different points in the visual field, biases can vary idiosyncratically across the spatial field as well. Here we ask whether variations in biases across space follow any regular pattern, and whether spatial and temporal variations are independent of one another. Measuring biases for surface orientation in structure-from-motion stimuli, sampled at numerous points in space and time, we find that variations both in time and space are positively autocorrelated: the closer two points are to each other in space or in time, the more similar the biases at those two points. We also found that spatial and temporal variations of bias are correlated, both between and within participants. Bias variations over space, time, and space-time are therefore not random but follow dynamics that may provide clues about the underlying mechanisms.

Numerosity adaptation (the underestimation of number after exposure to a numerous adaptor) is reduced when adaptor and test differ in color, suggesting that the numerosity system parses items into color-defined categories. Here we ask whether this chromatic selectivity is organized into multiple narrowly tuned chromatic channels, and whether its expression depends on individual chromatic sensitivity. Twenty observers (aged 22-61) completed two psychophysical tasks. First, chromatic discrimination was measured for five hues spaced in 5{degrees} CIE L*a*b* steps ({Delta}H = 0{degrees}, 5{degrees}, 10{degrees}, 15{degrees}, 20{degrees}) from a red reference (LCh: 54, 118, 38), yielding an individual just-noticeable difference (JND). Second, numerosity adaptation was measured across the same five chromatic distances between a 48-dot adaptor and the test. Observers with superior discrimination (JND < 2.5{degrees}) showed robust chromatic tuning, adaptation declining as the test moved away from the adaptor hue, whereas poorer discriminators showed none. Using an interindividual-covariance / factor-analytic approach, we found that adaptation strengths at neighboring chromatic distances were highly correlated and fell off with chromatic separation. Principal component analysis extracted two factors, one loading on the larger chromatic distances and one on the smaller; under oblique (promax) rotation the two factors were substantially correlated (r = .66), implying at least two dissociable but overlapping chromatically tuned mechanisms. These results suggest that numerosity adaptation is mediated by multiple, comparatively narrow chromatic channels, resembling the higher-order color mechanisms inferred from color scaling, SSVEP, and fMRI, rather than the two early cardinal axes (L-M, S-(L+M)).

The visual system optimizes its signal processing properties to efficiently encode natural scenes. The superior colliculus (SC) and primary visual cortex (V1) both play important roles in visual-motor processing, and they both exhibit qualitatively similar visual responses. However, it is not clear whether the SC simply inherits V1s efficient coding image processing optimizations or not, especially given that the SC receives a substantial amount of direct anatomical inputs from V1. Here, by performing matched experiments in the two brain areas, as well as with the same visual stimuli and in the same experimental animals, we show that the dynamics of coarse-to-fine visual image processing in the SC are much stronger than those in V1. In the SC, visual response latencies, being fastest for coarse patterns, are dictated by image spatial frequency, independently of the spatial frequency tuning curves of the neurons. On the other hand, V1 visual response latencies are largely dominated by visual response sensitivity, which is itself much more broadband than in the SC. These observations remain fundamentally unchanged in active vision gaze-shift scenarios eliciting visual reafferent responses in both brain areas. Our results suggest that coarse-to-fine visual image processing dynamics are most observable, and thus most relevant, in visually-responsive neurons driving foveating eye movements, like in the SC. Besides explaining behavioral evidence for saccadic facilitation by images that are consistent with the statistics of natural scenes, these results indicate that coarse-to-fine visual processing dynamics are much more of a collicular than a cortical visual phenomenon.

Humans rely on multiple sensory modalities, such as vision, audition, and touch, to perceive materials in everyday life. Previous research shows that multisensory perception leads to facilitation, yet the mechanisms responsible for this facilitation remain poorly understood. One potential mechanism is crossmodal prediction, whereby input from one modality generates predictions about another. While substantial research on multisensory facilitation has focused on bottom-up processes, such as spatial, temporal, and semantic congruency, the role of crossmodal predictions, particularly in material perception, has received little attention. To address this gap, we conducted two experiments, a reaction time task and a material rating task, in which participants viewed computer-generated animations of familiar objects being dropped to the ground. The paradigm exploited the natural temporal structure of impact events: pre-impact visual appearance provides information about an objects material and therefore can generate expectations about the forthcoming impact sound. Critically, participants saw the event only until before the impact, after which the video was masked. Thus, vision and audition were temporally aligned but not presented concurrently, allowing us to isolate the influence of visually driven expectations on the incoming auditory information without a bottom-up conflict. In some trials, the sound matched the expected material, but in a subset, it was incongruent, violating expectations elicited by the preceding visual information. Across both experiments, participants took longer to respond on incongruent than congruent trials, suggesting increased processing demands. In the rating task, incongruent trials also shifted material judgments, such that ratings reflected a weighted combination of incoming auditory information and visually driven predictions, with large individual differences in relative cue weighting. These findings suggest that priors on material properties from one modality, specifically vision, not only establish high-level expectations within the modality about an objects future state, but also extend across modalities.

Attentional control balances proactive suppression of predictable distractors with reactive suppression of unexpected ones. Yet, how internal states such as alertness shape this balance is unclear. Using pupillometry and eye tracking across two probability-cueing experiments (conducted in 2024) with varying distractor prevalence, we distinguished tonic (baseline pupil size across blocks) from trial-level pupil size fluctuations (trial-by-trial residual variability in pre-stimulus pupil size). With moderate prevalence, suppression of frequent-region distractors developed gradually, whereas high prevalence induced near-immediate suppression. Behavioral measures (e.g., reaction times) were closely linked to tonic and trial-level pupil size fluctuations. Critically, both alertness components jointly influenced control: during early learning, heightened trial-level pupil size increased distractor capture and reduced target fixations, whereas later on, suppression shifted to a proactive mode resilient to trial-level fluctuations. Under high prevalence, this shift occurred faster. Notably, higher trial-level pupil size generally accelerated first target selection. These findings show that tonic alertness and trial-level alertness fluctuations dynamically regulate reactive and proactive control during statistical learning. Impact StatementThis study shows that people become better at ignoring predictable distractions over time, but that this improvement depends not only on what they have learned about the task environment, but also on their current level of alertness. By combining eye tracking and pupil measures, we found that temporary increases in alertness can sometimes help people orient more quickly to relevant information, yet during earlier stages of learning they can also make attention more vulnerable to distracting events. These findings suggest that successful focus in complex environments depends on a dynamic interplay between learned expectations and moment-to-moment fluctuations in mental state, with implications for understanding sustained attention in settings such as monitoring, driving, and other tasks that require people to stay engaged while resisting distraction.

Eye contact is one of the most potent social signals, but it remains unclear how little visual input is sufficient to register direct gaze, and whether such registration requires conscious awareness. Here, we used a custom tachistoscope to present faces for only 1-5 ms, asking whether gaze direction can shape perception at the very limits of vision. Direct-gaze faces required less visual input to localise than averted-gaze faces and produced higher localisation sensitivity from 3 ms onwards. Crucially, this advantage emerged before participants could explicitly categorise gaze direction, and before perceptual awareness ratings showed metacognitive access to the information driving localisation, both of which appeared only at 4-5 ms. Information-theoretic analyses confirmed that localisation responses carried stimulus-location information before explicit reports and awareness ratings became informative. A second experiment showed that the earliest eye-contact advantage depended mainly on low spatial frequencies, indicating a role for coarse visual information. Finally, autistic traits were associated with a reduced localisation advantage for direct gaze. These findings show that eye contact can influence perceptual processing within just a few milliseconds, with less visual stimulation than is required for conscious access to gaze direction.

Purpose: To evaluate the feasibility, validity, and scalability of PLFest, an open-source, Unity-based, cross-platform application designed for standardized, multi-site visual and cognitive assessment and training. Methods: Two hundred sixty participants (mean age = 23 years) were recruited across four university sites in the United States. Participants completed a battery of five visual assessments administered through PLFest, including visual acuity, contrast sensitivity, spatial frequency cutoff, contrast sensitivity at spatial-frequency cutoff, and visual search. Five cognitive assessments measuring visuospatial working memory, verbal working memory, fluid reasoning, inhibitory control, and selective attention were also administered. Descriptive statistics and performance distributions were examined and compared with normative data. Results: Visual acuity and contrast sensitivity measures closely matched previously reported normative values obtained using established clinical and psychophysical methods. Spatial frequency cutoff and visual search tasks produced stable threshold estimates while showing substantial inter-individual variability. Performance across all cognitive assessments was consistent with published validation studies of the corresponding tasks. Across the full battery, adaptive procedures demonstrated reliable convergence and generated well-distributed performance measures without evidence of substantial floor or ceiling effects. Importantly, these findings were observed across four geographically distributed testing sites using standardized consumer-grade tablet hardware. Conclusions: PLFest provides reliable and scalable assessment of visual and cognitive function using portable consumer devices. The platform supports standardized data collection across distributed research settings while maintaining performance characteristics consistent with established laboratory and clinical benchmarks. These findings support the use of PLFest as a reliable framework for large-scale studies of vision and cognition. Translational Relevance: By reducing dependence on specialized laboratory infrastructure and trained personnel, PLFest may facilitate broader access to visual and cognitive assessment, enabling large-scale research, screening, and future rehabilitation applications.

Filling-in occurs during the perceptual disappearance of a blank figure presented on a textured background. Current models of perceptual filling-in are based on a two-stage model where the figure boundary weakens after a period of adaptation, followed by the spreading of the background representation into the region representing the figure. This suggests a competition between figure boundary and background representations whereby filling-in is facilitated by a weaker boundary representation and a stronger background representation. Here, we test this interpretation, by using the oblique effect and surround-modulation suppression, which are functional properties of early visual cortex that modulate the expected strengths of the responses to the background texture and to the figure boundary. In a sample of N=58 participants, we found more filling-in with background textures of cardinal compared to oblique orientations (earlier onset time, with more and longer episodes of filling-in per trial), in line with a known, stronger neuronal response for cardinal than for oblique orientation in early visual cortex. We found more filling-in when the main axis of the rectangular figure was iso-oriented rather than cross-oriented with the background texture (more and longer episodes of filling-in per trial, but no change in onset time), in line with a lower response to oriented stimuli when surrounded by iso-oriented flankers compared to cross-oriented flankers. Overall, our results support the two-stage model and suggest the involvement of early visual cortical areas characterized by the oblique effect and orientation- tuned surround-suppression.

Visual search is a critical component of many professions such as military operations, baggage screening, and radiology. Aided Target Recognition (AiTR) systems are designed to highlight potential threats across the operator visual field in real-time, directing attention and improving accuracy. However, these systems may impact search and, consequently, situational awareness by diverting attentional resources from non-highlighted, yet relevant, locations. Previous work suggests that scene gist is extracted within the first 250 ms of scene onset (Vo & Henderson, 2010). As such, this study examined whether a 250 ms AiTR onset delay could encourage a more even distribution of attention. Participants searched synthetically generated scenes and classified each person in the scene as armed or unarmed. Depending on their condition, participants either saw the scenes unaugmented (No AiTR condition), with AiTR highlights consisting of red bounding boxes around armed people and yellow boxes around unarmed (AiTR condition), or with AiTR highlights presented 250 ms post scene onset (Delayed AiTR condition). A surprise memory test of background objects presented in the search scenes was administered to all participants upon completion of the search task. As predicted and preregistered, results showed less overt attentional deployment to background information (anything other than the people themselves) in the AiTR condition compared to No AiTR , however, decreased overt attentional deployment was not seen in the Delayed AiTR group. A similar pattern was observed in the memory data (with the AiTR condition having a lower score than the No AiTR condition and the Delayed AiTR condition), this difference was not significant.

Social media users often feel that time vanishes while scrolling, but real feeds confound novelty, rewards, social signals, and self-paced control, leaving the driver of this distortion unclear. We tested whether self-paced visual exploration is sufficient to compress subjective time by comparing active scrolling with passive, yoked viewing and a static baseline. Twenty-three adults viewed sequences of natural images under three within-subject conditions: Scrolling (self-paced mouse clicks), Watching (a passive, yoked replay of their own scrolling sequence), and a Baseline (a static image). Participants estimated the elapsed duration of each block. Subjective duration was most compressed under Scrolling (48% of elapsed time), followed by Watching (51%) and Baseline (65%). Two sources separated these effects. Adding back the empty inter-image fixations brought the image-rich conditions to within seconds of the Baseline, showing that observers barely counted the blank gaps; the Scrolling--Watching difference, by contrast, was independent of these shared gaps, isolating self-paced control as a second source of compression. Electrophysiology linked that control to anticipatory neural states and the timing of early visual responses, with no amplified encoding of individual images. The results favor an attention-weighted account of timing, on which subjective duration tracks how much attention reaches the clock, a resource that a self-paced stream and its uncounted gaps both draw away.

Object representations in the human ventral occipitotemporal cortex (VOTC) are organized along multiple dimensions, including shape (rectilinear vs. curvilinear), real-world size (large vs. small), and mobility (stationary vs. mobile). However, these dimensions are strongly correlated in naturalistic vision, making their separate contributions to VOTC organization unclear. For example, large objects (e.g., a wardrobe, a house) are typically rectilinear and stationary, while small objects (e.g., a ball, a cup) are more curvilinear and mobile. Here, we used fMRI, together with a new stimulus set that orthogonally manipulates shape, size, and mobility, to investigate the separate influences of these dimensions on VOTC organization. Example stimuli include air balloon (large, curvilinear, mobile), radar dish (large, curvilinear, stationary), and mailbox (small, rectilinear, stationary). Contrasts revealed that large (vs. small), rectilinear (vs. curvilinear), and stationary (vs. mobile) dimensions all independently evoked strong and overlapping activity in medio-anterior VOTC. This overlapping activity was at the intersection of the parahippocampal place area (PPA) and the ventral place-memory area (VPMA). Similar results were found at the intersection of the scene-selective occipital place area and the lateral place-memory area (LPMA). Finally, large (vs. small), but not rectilinear (vs. curvilinear) or stationary (vs. mobile) activity, was found in additional posterior ventral scene-selective regions, as well as in early visual cortex. Overall, these results indicate that object shape, real-world size, and mobility dimensions all independently activate scene-selective PPA and OPA, showing joint selectivity for distinct low- and high-level object properties that are highly correlated in naturalistic vision.

Interception of moving objects requires the nervous system to compensate for sensory delays and uncertainty, yet how behavior is controlled remains debated. Key questions concern whether predictive processes play any role at all and, if so, whether they rely on simple motion extrapolation or incorporate internalized physical priors, such as gravity. Another open question is whether observers adopt a single control strategy or flexibly switch between predictive and reactive control - or between different predictive strategies - depending on task demands. To address these questions, we developed a virtual interception task in which participants intercepted moving targets under systematically varied conditions. We manipulated gravity (1g vs. 0g), visual availability (occluded vs. non-occluded), target velocity, and the initial spatial configuration of the ball and paddle (same vs. opposite side). Results indicate that interception is supported by predictive mechanisms across conditions. Behavioral patterns during occluded 0g trials suggest that participants extrapolate target motion using expectations consistent with gravity. Target velocity, visual occlusion, and task geometry modulated movement strategies, indicating that predictive control is flexibly adapted to task demands. These findings support the view that interception relies on predictive internal models incorporating structured physical priors while revealing flexible, context-dependent adaptations to sensory and task constraints.